Method and system for device control

ABSTRACT

The invention concerns device control for enabling electrical or electronic devices to be controlled through computer systems. Employing the invention involves the use of one or more control interfaces which are capable of being used to interacting with a computer system or network, each interface typically comprising sheet material such as paper which has coded data disposed on it and which allows it to be used to interact with the computer system by use of a sensing device operated by a user. Through this interaction, data can be forwarded to the computer system from the control interface to effect in that system an operation relating to the functioning of the device.

FIELD OF INVENTION

The present invention relates generally to device control and, moreparticularly, to a method and system for enabling electrical orelectronic devices to be controlled through distributed computingsystems. It has specific application to the operation of a deviceinvolving a printed form-based user interface.

CO-PENDING APPLICATIONS

Various methods, systems and apparatus relating to the present inventionare disclosed in the following co-pending applications/granted patentsfiled by the applicant or assignee of the present invention on Oct. 20,2000:

Ser. Nos. 09/693,415, 09/693,219, U.S. Pat. No. 6,813,558, Ser. No09/693,515, U.S. Pat. No. 6,847,883, Ser. Nos. 09/693,593, 09/693,647,09/693,690, U.S. Pat. Nos. 6,474,888, 6,627,870, 6,724,374, Ser. Nos.09/693,514, 09/693,301, U.S. Pat. Nos. 6,454,482, 6,808,330, 6,527,365,6,474,773, 6,550,997The disclosures of these co-pending applications are incorporated hereinby reference.

Various methods, systems and apparatus relating to the present inventionare disclosed in the following co-pending applications/granted patentsfiled by the applicant or assignee of the present invention on Sep. 15,2000:

U.S. Pat. No. 6,679,420, Ser. Nos. 09/663,599, 09/663,701, U.S. Pat. No.6,720,985

The disclosures of these co-pending applications are incorporated hereinby reference.

Various methods, systems and apparatus relating to the present inventionare disclosed in the following co-pending applications/granted patentsfiled by the applicant or assignee of the present invention on Jun. 30,2000:

U.S. Pat. No. 6,824,044, Ser. Nos. 09/607,852, 09/608,970, U.S. Pat. No.6,678,499, Ser. No. 09/607,656, U.S. Pat. No. 6,766,942, Ser. Nos.09/609,303, 09/610,095, 09/609,596, 09/607,843, 09/607,605, 09/608,178,09/609,553, 09/609,233, 09/609,149, 09/608,022, 09/609,232, 09/607,844,U.S. Pat. Nos. 6,457,883, 6,831,682, Ser. No. 09/607,985, U.S. Pat. Nos.6,398,332, 6,394,573, 6,622,923The disclosures of these co-pending applications are incorporated hereinby reference.

Various methods, systems and apparatus relating to the present inventionare disclosed in the following co-pending applications/granted patentsfiled by the applicant or assignee of the present invention on 23 May2000:

-   -   Ser. Nos. 09/575,197, 09/575,195, 09/575,159, 09/575,132,        09/575,123, U.S. Pat. No. 6,825,945, Ser. Nos. 09/575,130,        09/575,165, U.S. Pat. No. 6,813,039, Ser. Nos. 09/575,116,        09/575,118, 09/575,131, U.S. Pat. Nos. 6,816,274, 6,681,045,        Ser. Nos. 09/575,139, 09/575,186, U.S. Pat. No. 6,728,000, Ser.        Nos. 09/575,181, 09/575,145, 09/575,192, 09/575,193, 09/575,156,        09/575,183, U.S. Pat. No. 6,789,194, Ser. No. 09/575,150, U.S.        Pat. Nos. 6,789,191, 6,644,642, 6,502,614, 6,622,999, 6,669,385,        6,549,935, Ser. No. 09/575,187, U.S. Pat. Nos. 6,727,996,        6,760,119, 6,591,884, 6,439,706, Ser. No. 09/575,198, U.S. Pat.        Nos. 6,290,349, 6,428,155, 6,785,016, 6,870,966, Ser. No.        09/575,154, U.S. Pat. Nos. 6,822,639, 6,737,591, Ser. No.        09/575,129, U.S. Pat. Nos. 6,830,196, 6,832,717, Ser. Nos.        09/575,189, 09/575,162, 09/575,172, 09/575,170, 09/575,171,        09/575,161, U.S. Pat. Nos. 6,428,133, 6,527,365 6,315,399,        6,338,548, 6,540,319, 6,328,431, 6,328,425, Ser. No. 09/575,127,        U.S. Pat. Nos. 6,383,833, 6,464,332, 6,390,591, Ser. No.        09/575,152, U.S. Pat. Nos. 6,328,417, 6,409,323, 6,281,912,        6,604,810, 6,318,920, 6,488,422, 6,795,215, Ser. No. 09/575,109        U.S. Pat. No. 6,859,289        The disclosures of these co-pending applications are        incorporated herein by reference.

BACKGROUND

There are many electrical and electronic devices in the home and officewhich have some kind of user interface for controlling their operation.These include office equipment, audio and video equipment, kitchenappliances, heating and cooling systems, etc. Even light sockets haveon/off switches and sometimes dimmers, and in many countries poweroutlets have on/off switches.

Office equipment and entertainment equipment is increasinglynetworkable—i.e. devices come with interfaces to standard local-areanetworks, allowing their operation to be controlled over a network. Theadvent of standard wireless networking technologies such as Bluetoothmay see an increase in networking support in such devices andappliances.

Some networking futurists predict that even light sockets will one daybe networked, allowing their status to be queried and their function tobe controlled remotely.

Even today, an increasing number of devices can be controlled viainfrared remote control interfaces. Such interfaces are particularlyprevalent on audio and video equipment and heating and cooling systems.

SUMMARY OF INVENTION

It is an objective of the present invention to provide a new method andsystem for enabling devices to be controlled through a computer system.

In a first aspect, the present invention provides for a method ofenabling a device to be controlled in which the device is operative toperform at least one function in response to control instructions from acomputer system, the method including the steps of

providing a control interface containing information relating to thedevice function, the control interface including coded data indicativeof an identity of the control interface and of at least one referencepoint of the control interface;

receiving, in the computer system, indicating data from a sensing deviceregarding the identity of the control interface and a position of thesensing device relative to the control interface, the sensing device,when placed in an operative position relative to the control interface,sensing the indicating data using at least some of the coded data; and

effecting, in the computer system and from the indicating data, anoperation relating to at least one parameter of the controlinstructions.

In a second aspect, the present invention provides for a method ofenabling a device to be controlled in which the device is operative toperform at least one function in response to control instructions from acomputer system, the method including the steps of

providing a control interface containing information relating to thedevice function, the control interface including coded data indicativeof at least one parameter of the control instructions;

receiving, in the computer system, data from a sensing device relatingto said at least one parameter and regarding movement of the sensingdevice relative to the control interface, the sensing device, when movedrelative to the control interface, sensing the data regarding said atleast one parameter using at least some of the coded data and generatingthe data regarding its own movement relative to the control interface;and

interpreting, in the computer system, said movement of the sensingdevice as it relates to said at least one parameter.

In a third aspect, the present invention provides for a method ofenabling a device to be controlled in which the device is operative toperform at least one function in response to control instructions from acomputer system, the method including the steps of

providing a user with a control interface including coded dataindicative of an identity of the control interface;

receiving, in the computer system, data from a sensing device regardingan identity of the user and regarding the identity of the controlinterface, the sensing device containing the data regarding the identityof the user and sensing the data regarding the identity of the controlinterface using at least some of the coded data; and

effecting, in the computer system and from the data regarding theidentity of the user and the identity of the control instruction, anoperation relating to at least one parameter of the controlinstructions.

In a fourth aspect, the present invention provides for a system forenabling at least one function of a device to be controlled, the systemincluding

a control interface containing information relating to the devicefunction, the control interface including coded data indicative of anidentity of the control interface and of at least one reference point ofthe control interface; and

a computer system incorporating control instructions operative to causesaid device to perform said function and operative to receive indicatingdata from a sensing device for effecting an operation relating to atleast one parameter of the control instructions, the indicating databeing indicative of the identity of the control interface and a positionof the sensing device relative to the control interface, the sensingdevice sensing the indicating data using at least some of the codeddata.

In a fifth aspect, the present invention provides for a system forenabling at least one function of a device to be controlled in responseto control instructions, the system including

a control interface containing information relating to the devicefunction, the control interface including coded data indicative of atleast one parameter of the control instructions; and

a computer system incorporating said control instructions, and operativeto receive data from a sensing device regarding said at least oneparameter and regarding movement of the sensing device relative to thecontrol interface, and for interpreting said movement of the sensingdevice as it relates to said at least one parameter, the sensing device,when moved relative to the control interface, sensing the data regardingsaid at least one parameter using at least some of the coded data andgenerating the data regarding its own movement relative to the controlinterface.

In a sixth aspect, the present invention provides for a system forenabling at least one function of a device to be controlled, the systemincluding

a control interface including coded data indicative of an identity ofthe control interface; and

a computer system incorporating control instructions to cause saiddevice to perform said function, and operative to receive from a sensingdevice data regarding an identity of the user and the identity of thecontrol interface, and for effecting, from said received data, anoperation relating to a parameter of said control instructions, thesensing device containing the data regarding the identity of the userand sensing the data regarding the identity of the control interfaceusing at least some of the coded data.

Accordingly, the present invention provides a system and a method whichutilizes one or more device control interfaces capable of interactingwith a computer system to control at least one function of at least onedevice. Whilst the novel method and system of the present invention maybe used in conjunction with a single computer system, in a particularlypreferred form it is designed to operate over a computer network, suchas the Internet.

Physically, the device control interface is disposed on a surface mediumwhich may be of any suitable structure. However, in a preferredarrangement, the form comprises sheet material such as paper or the likewhich has the coded data printed on it and which allows it to interactwith the computer system. The coded data is detectable preferably, butnot exclusively, outside the visible spectrum, thereby enabling it to bemachine-readable but substantially invisible to the human eye. Thedevice control may also include visible material which providesinformation to a user, such as the application or purpose of the devicecontrol, and which visible information may be registered or correlate inposition with the relevant hidden coded data.

The system also includes a sensing device to convey data from the devicecontrol to the computer system, and in some instances, to contributeadditional data. Again, the sensing device may take a variety of formsbut is preferably compact and easily portable. In one arrangement, thesensing device is configured as a pen which is designed to be able tophysically mark the interactive form as well as to selectively enablethe coded data from the form to be read and transmitted to the computersystem. The coded data then provides control information, configuredsuch that designation thereof by a user causes instructions to beapplied to the software running on the computer system or network.Alternatively, the sensing device may be embedded in a ring, hand gloveor finger glove worn by a user. It could also be embedded in customeyewear, with interaction controlled by gaze.

The nature of the interaction between the device control interface andthe sensing device and the data that each contributes to the computersystem may vary. In one arrangement, the coded data on the form isindicative of the identity of the form and of at least one referencepoint on that form. In another embodiment, the interactive form includescoded data which is indicative of a parameter of the form, whereas thesensing device is operative to provide data regarding its own movementrelative to that form to the computer system together with coded datafrom the form. In yet another arrangement, the form includes the codeddata which at least identifies the form, and the sensing device isdesigned to provide, to the computer system, data based on the formcoded data, and also on data which identifies the user of the device.

The system and method also preferably employs specially designedprinters to print the control devices. Further these printers constituteor form part of the computer system and are designed to receive datafrom the sensing device. As indicated above, the system and method ofthe invention is ideally suited to operate over a network. In thisarrangement, the printers are fully integrated into the network andallow for printing of the interactive forms on demand and also fordistributing of the forms using a mixture of multi-cast and point-castcommunication protocols.

The present invention therefore provides a novel system and method forenabling electrical or electronic devices to be controlled through acomputer system making use of novel interactive surface media. Theinvention is applicable where a particular device is able to be remotelycontrolled, either via a network interface or an infrared interface. Inthese circumstaces it is possible to completely separate the userinterface to the device from the device itself and the user interfacecan then be implemented in the forms mentioned above and sensed by asensing device carried by the user.

The device control interface preferably identifies the device, and isprinted with the user interface of the device in the form of buttonsetc. Being passive, the surface gives no feedback to the user as to thestatus of the device or its user interface. Feedback can, however, beprovided via the sensing device if necessary. The computer systemresolves operations on the control interface with reference to a userinterface structure previously selected by the user from the deviceitself or its manufacturer, or designed by the user using elementsprovided by the device or its manufacturer.

User interfaces implemented in this way have several advantages:

once the infrastructure is in place, the cost of manufacturing the userinterface is removed from the cost of manufacturing each particulardevice; as an example, even the cost of the traditional user interfaceto a light socket (i.e. both the wiring and the switch) is non-trivial;

a user interface can be replicated by simply printing another copy

when a user interface becomes damaged it can be easily replaced

when the firmware in a device is updated, a new user interface can beeasily provided

a user interface can be customized by arranging elements and printingthe result

a user interface with a particular style can be chosen from an availableset; for example, some users may prefer larger icons or a simplifiedfunction set

multiple user interfaces can be grouped together

a user interface can be provided on the device itself by the same means;i.e. pre-printed and attached to the device, or printed directly ontothe device surface

access to individual user interfaces and functions can be controlled,since each sensing device is tied to a particular user's identity.

As one example, a device control interface according to a preferredaspect of the invention can be printed onto or attached to the surfaceof a toy to provide the user interface to the toy. An appropriatesensing device can then be used to interact with the user interface,communicating by wire or wirelessly to either a personal computer, gameconsole, or a processor embedded in the toy itself, to produce graphicaloutput, sound effects and other behavior. For example, a toy sit-insubmarine may have a large user interface containing dozens of submarinecontrols. The toy comes with a CD-ROM containing an associatedinteractive game which runs on a PC or game console. The game is drivenby the user interface contained in the submarine, perhaps mediated by asensing device in the form of a glove which communicates wirelessly byBluetooth with standard Bluetooth hardware in the PC or game console.The PC screen or console-connected television monitor provides theanimated part of the user interface. The device control interfaceprovides the extended user interface. Toys can be customized byselecting different user interfaces and custom user interfaces can becreated by assembling pre-defined user interface components.

BRIEF DESCRIPTION OF DRAWINGS

Preferred and other embodiments of the invention will now be described,by way of non-limiting example only, with reference to the accompanyingdrawings, in which:

FIG. 1 is a schematic of a the relationship between a sample printednetpage and its online page description;

FIG. 2 is a schematic view of a interaction between a netpage pen, anetpage printer, a netpage page server, and a netpage applicationserver;

FIG. 3 is a schematic view of a high-level structure of a printednetpage and its online page description;

FIG. 4 a is a plan view showing a structure of a netpage tag;

FIG. 4 b is a plan view showing a relationship between a set of the tagsshown in FIG. 4 a and a field of view of a netpage sensing device in theform of a netpage pen;

FIG. 5 a is a plan view showing an alternative structure of a netpagetag;

FIG. 5 b is a plan view showing a relationship between a set of the tagsshown in FIG. 5 a and a field of view of a netpage sensing device in theform of a netpage pen;

FIG. 5 c is a plan view showing an arrangement of nine of the tags shownin FIG. 5 a where targets are shared between adjacent tags;

FIG. 5 d is a plan view showing the interleaving and rotation of thesymbols of the four codewords of the tag shown in FIG. 5 a;

FIG. 6 is a schematic view of a set of user interface flow documenticons;

FIG. 7 is a schematic view of a set of user interface page layoutelement icons;

FIG. 8 is a schematic view of device control interactions;

FIG. 9 is a schematic view of a device control class diagram;

FIG. 10 is a schematic view of a default device class diagram;

FIG. 11 is a schematic view of a device information class diagram;

FIG. 12 is a schematic view of a device channel class diagram;

FIG. 13 is a schematic view of a remote control class diagram;

FIG. 14 is a schematic view of a program schedule class diagram;

FIG. 15 is a schematic view of a device setup user interface flow;

FIG. 16 is a schematic view of a setup details user interface flow;

FIG. 17 is a schematic view of a device list user interface flow;

FIG. 18 is a schematic view of an add device user interface flow;

FIG. 19 is a schematic view of a record program user interface flow;

FIG. 20 is a detailed view of a device setup main page;

FIG. 21 is a schematic view of a device setup details;

FIG. 22 is a schematic view of a device list;

FIG. 23 is a schematic view of a universal remote control form;

FIG. 24 is a schematic view of a device specific TV remote control;

FIG. 25 is a schematic view of a universal remote control;

FIG. 26 is a schematic view of a select manufacturer list;

FIG. 27 is a schematic view of a select device type list;

FIG. 28 is a schematic view of a select model list;

FIG. 29 is a schematic view of a set defaults form; and

FIG. 30 is a schematic view of a program schedule.

DETAILED DESCRIPTION OF PREFERRED AND OTHER EMBODIMENTS

Note: Memjet™ is a trademark of Silverbrook Research Pty Ltd, Australia.

In the preferred embodiment, the invention is configured to work withthe netpage networked computer system, a summary of which is given belowand a detailed description of which is given in our earlierapplications, including in particular applications U.S. Ser. No.09/575,129, U.S. Ser. No. 09/575,171, U.S. Ser. No. 09/575,155, U.S.Ser. No. 09/575,195 and U.S. Ser. No. 09/575,141. It will be appreciatedthat not every implementation will necessarily embody all or even mostof the specific details and extensions described in these applicationsin relation to the basic system. However, the system is described in itsmost complete form to assist in understanding the context in which thepreferred embodiments and aspects of the present invention operate.

In brief summary, the preferred form of the netpage system employs acomputer interface in the form of a mapped surface, that is, a physicalsurface which contains references to a map of the surface maintained ina computer system. The map references can be queried by an appropriatesensing device. Depending upon the specific implementation, the mapreferences may be encoded visibly or invisibly, and defined in such away that a local query on the mapped surface yields an unambiguous mapreference both within the map and among different maps. The computersystem can contain information about features on the mapped surface, andsuch information can be retrieved based on map references supplied by asensing device used with the mapped surface. The information thusretrieved can take the form of actions which are initiated by thecomputer system on behalf of the operator in response to the operator'sinteraction with the surface features.

In its preferred form, the netpage system relies on the production of,and human interaction with, netpages. These are pages of text, graphicsand images printed on ordinary paper or other media, but which work likeinteractive web pages. Information is encoded on each page using inkwhich is substantially invisible to the unaided human eye. The ink,however, and thereby the coded data, can be sensed by an opticallyimaging pen and transmitted to the netpage system.

In the preferred form, active buttons and hyperlinks on each page can beclicked with the pen to request information from the network or tosignal preferences to a network server. In one embodiment, text writtenby hand on a netpage is automatically recognized and converted tocomputer text in the netpage system, allowing forms to be filled in. Inother embodiments, signatures recorded on a netpage are automaticallyverified, allowing e-commerce transactions to be securely authorized.

As illustrated in FIG. 1, a printed netpage 1 can represent ainteractive form which can be filled in by the user both physically, onthe printed page, and “electronically”, via communication between thepen and the netpage system. The example shows a “Request” formcontaining name and address fields and a submit button. The netpageconsists of graphic data 2 printed using visible ink, and coded data 3printed as a collection of tags 4 using invisible ink. The correspondingpage description 5, stored on the netpage network, describes theindividual elements of the netpage. In particular it describes the typeand spatial extent (zone) of each interactive element (i.e. text fieldor button in the example), to allow the netpage system to correctlyinterpret input via the netpage. The submit button 6, for example, has azone 7 which corresponds to the spatial extent of the correspondinggraphic 8.

As illustrated in FIG. 2, the netpage pen 101, a preferred form of whichis described in our earlier application U.S. Ser. No. 09/575,174, worksin conjunction with a netpage printer 601, an Internet-connectedprinting appliance for home, office or mobile use. The pen is wirelessand communicates securely with the netpage printer via a short-rangeradio link 9.

The netpage printer 601, preferred forms of which are described in ourearlier application U.S. Ser. No. 09/575,155 and our co-filedapplication U.S. Ser. No. 09/693,514, is able to deliver, periodicallyor on demand, personalized newspapers, magazines, catalogs, brochuresand other publications, all printed at high quality as interactivenetpages. Unlike a personal computer, the netpage printer is anappliance which can be, for example, wall-mounted adjacent to an areawhere the morning news is first consumed, such as in a user's kitchen,near a breakfast table, or near the household's point of departure forthe day. It also comes in tabletop, desktop, portable and miniatureversions.

Netpages printed at their point of consumption combine the ease-of-useof paper with the timeliness and interactivity of an interactive medium.

As shown in FIG. 2, the netpage pen 101 interacts with the coded data ona printed netpage 1 and communicates, via a short-range radio link 9,the interaction to a netpage printer. The printer 601 sends theinteraction to the relevant netpage page server 10 for interpretation.In appropriate circumstances, the page server sends a correspondingmessage to application computer software running on a netpageapplication server 13. The application server may in turn send aresponse which is printed on the originating printer.

The netpage system is made considerably more convenient in the preferredembodiment by being used in conjunction with high-speedmicroelectromechanical system (MEMS) based inkjet (Memjet™) printers,for example as described in our earlier application U.S. Ser. No.09/575,141. In the preferred form of this technology, relativelyhigh-speed and high-quality printing is made more affordable toconsumers. In its preferred form, a netpage publication has the physicalcharacteristics of a traditional newsmagazine, such as a set ofletter-size glossy pages printed in full color on both sides, boundtogether for easy navigation and comfortable handling.

The netpage printer exploits the growing availability of broadbandInternet access. The netpage printer can also operate with slowerconnections, but with longer delivery times and lower image quality. Thenetpage system can also be enabled using existing consumer inkjet andlaser printers, although the system will operate more slowly and willtherefore be less acceptable from a consumer's point of view. In otherembodiments, the netpage system is hosted on a private intranet. Instill other embodiments, the netpage system is hosted on a singlecomputer or computer-enabled device, such as a printer.

Netpage publication servers 14 on the netpage network are configured todeliver print-quality publications to netpage printers. Periodicalpublications are delivered automatically to subscribing netpage printersvia pointcasting and multicasting Internet protocols. Personalizedpublications are filtered and formatted according to individual userprofiles.

A netpage printer can be configured to support any number of pens, and apen can work with any number of netpage printers. In the preferredimplementation, each netpage pen has a unique identifier. A householdmay have a collection of colored netpage pens, one assigned to eachmember of the family. This allows each user to maintain a distinctprofile with respect to a netpage publication server or applicationserver.

A netpage pen can also be registered with a netpage registration server11 and linked to one or more payment card accounts. This allowse-commerce payments to be securely authorized using the netpage pen. Thenetpage registration server compares the signature captured by thenetpage pen with a previously registered signature, allowing it toauthenticate the user's identity to an e-commerce server. Otherbiometrics can also be used to verify identity. A version of the netpagepen includes fingerprint scanning, verified in a similar way by thenetpage registration server.

Although a netpage printer may deliver periodicals such as the morningnewspaper without user intervention, it can be configured never todeliver unsolicited junk mail. In its preferred form, it only deliversperiodicals from subscribed or otherwise authorized sources. In thisrespect, the netpage printer is unlike a fax machine or e-mail accountwhich is visible to any junk mailer who knows the telephone number ore-mail address.

Each object model in the system is described using a Unified ModelingLanguage (UML) class diagram. A class diagram consists of a set ofobject classes connected by relationships, and two kinds ofrelationships are of interest here: associations and generalizations. Anassociation represents some kind of relationship between objects, i.e.between instances of classes. A generalization relates actual classes,and can be understood in the following way: if a class is thought of asthe set of all objects of that class, and class A is a generalization ofclass B, then B is simply a subset of A. Each class is drawn as arectangle labelled with the name of the class. It contains a list of theattributes of the class, separated from the name by a horizontal line,and a list of the operations of the class, separated from the attributelist by a horizontal line. In the class diagrams which follow, however,operations are never modelled. An association is drawn as a line joiningtwo classes, optionally labelled at either end with the multiplicity ofthe association. The default multiplicity is one. An asterisk (*)indicates a multiplicity of “many”, i.e. zero or more. Each associationis optionally labelled with its name, and is also optionally labelled ateither end with the role of the corresponding class. An open diamondindicates an aggregation association (“is-part-of”), and is drawn at theaggregator end of the association line. A generalization relationship(“is-a”) is drawn as a solid line joining two classes, with an arrow (inthe form of an open triangle) at the generalization end. When a classdiagram is broken up into multiple diagrams, any class which isduplicated is shown with a dashed outline in all but the main diagramwhich defines it. It is shown with attributes only where it is defined.

Netpages are the foundation on which a netpage network is built. Theyprovide a paper-based user interface to published information andinteractive services. A netpage consists of a printed page (or othersurface region) invisibly tagged with references to an onlinedescription of the page. The online page description is maintainedpersistently by a netpage page server. The page description describesthe visible layout and content of the page, including text, graphics andimages. It also describes the input elements on the page, includingbuttons, hyperlinks, and input fields. A netpage allows markings madewith a netpage pen on its surface to be simultaneously captured andprocessed by the netpage system.

Multiple netpages can share the same page description. However, to allowinput through otherwise identical pages to be distinguished, eachnetpage is assigned a unique page identifier. This page ID hassufficient precision to distinguish between a very large number ofnetpages.

Each reference to the page description is encoded in a printed tag. Thetag identifies the unique page on which it appears, and therebyindirectly identifies the page description. The tag also identifies itsown position on the page. Characteristics of the tags are described inmore detail below.

Tags are printed in infrared-absorptive ink on any substrate which isinfrared-reflective, such as ordinary paper. Near-infrared wavelengthsare invisible to the human eye but are easily sensed by a solid-stateimage sensor with an appropriate filter.

A tag is sensed by an area image sensor in the netpage pen, and the tagdata is transmitted to the netpage system via the nearest netpageprinter. The pen is wireless and communicates with the netpage printervia a short-range radio link. Tags are sufficiently small and denselyarranged that the pen can reliably image at least one tag even on asingle click on the page. It is important that the pen recognize thepage ID and position on every interaction with the page, since theinteraction is stateless. Tags are error-correctably encoded to makethem partially tolerant to surface damage.

The netpage page server maintains a unique page instance for eachprinted netpage, allowing it to maintain a distinct set of user-suppliedvalues for input fields in the page description for each printednetpage.

The relationship between the page description, the page instance, andthe printed netpage is shown in FIG. 3. The printed netpage may be partof a printed netpage document 45. The page instance is associated withboth the netpage printer which printed it and, if known, the netpageuser who requested it.

In a preferred form, each tag identifies the region in which it appears,and the location of that tag within the region. A tag may also containflags which relate to the region as a whole or to the tag. One or moreflag bits may, for example, signal a tag sensing device to providefeedback indicative of a function associated with the immediate area ofthe tag, without the sensing device having to refer to a description ofthe region. A netpage pen may, for example, illuminate an “active area”LED when in the zone of a hyperlink.

In a preferred embodiment, each tag contains an easily recognizedinvariant structure which aids initial detection, and which assists inminimizing the effect of any warp induced by the surface or by thesensing process. The tags preferably tile the entire page, and aresufficiently small and densely arranged that the pen can reliably imageat least one tag even on a single click on the page. It is importantthat the pen recognize the page ID and position on every interactionwith the page, since the interaction is stateless.

In a preferred embodiment, the region to which a tag refers coincideswith an entire page, and the region ID encoded in the tag is thereforesynonymous with the page ID of the page on which the tag appears. Inother embodiments, the region to which a tag refers can be an arbitrarysubregion of a page or other surface. For example, it can coincide withthe zone of an interactive element, in which case the region ID candirectly identify the interactive element.

Each tag contains typically contains 16 bits of tag ID, at least 90 bitsof region ID, and a number of flag bits. Assuming a maximum tag densityof 64 per square inch, a 16-bit tag ID supports a region size of up to1024 square inches. Larger regions can be mapped continuously withoutincreasing the tag ID precision simply by using abutting regions andmaps. The distinction between a region ID and a tag ID is mostly one ofconvenience. For most purposes the concatenation of the two can beconsidered as a globally unique tag ID. Conversely, it may also beconvenient to introduce structure into the tag ID, for example to definethe x and y coordinates of the tag. A 90-bit region ID allows 2⁹⁰ (˜10²⁷or a thousand trillion trillion) different regions to be uniquelyidentified. Tags may also contain type information, and a region may betagged with a mixture of tag types. For example, a region may be taggedwith one set of tags encoding x coordinates and another set, interleavedwith the first, encoding y coordinates.

In one embodiment, 120 bits of tag data are redundantly encoded using a(15, 5) Reed-Solomon code. This yields 360 encoded bits consisting of 6codewords of 15 4-bit symbols each. The (15, 5) code allows up to 5symbol errors to be corrected per codeword, i.e. it is tolerant of asymbol error rate of up to 33% per codeword. Each 4-bit symbol isrepresented in a spatially coherent way in the tag, and the symbols ofthe six codewords are interleaved spatially within the tag. This ensuresthat a burst error (an error affecting multiple spatially adjacent bits)damages a minimum number of symbols overall and a minimum number ofsymbols in any one codeword, thus maximising the likelihood that theburst error can be fully corrected.

Any suitable error-correcting code code can be used in place of a (15,5) Reed-Solomon code, for example a Reed-Solomon code with more or lessredundancy, with the same or different symbol and codeword sizes;another block code; or a different kind of code, such as a convolutionalcode (see, for example, Stephen B. Wicker, Error Control Systems forDigital Communication and Storage, Prentice-Hall 1995, the contents ofwhich a herein incorporated by cross-reference).

One embodiment of the physical representation of the tag, shown in FIG.4 a and described in our earlier application U.S. Ser. No. 09/575,129includes fixed target structures 15, 16, 17 and variable data areas 18.The fixed target structure allow a sensing device such as the netpagepen to detect the tag and infer its three-dimensional orientationrelative to the sensor. The data areas contain representations of theindividual bits of the encoded tag data. To maximise its size, each databit is represented by a radial wedge in the form of an area bounded bytwo radial lines and two concentric circular arcs. Each wedge has aminimum dimension of 8 dots at 1600 dpi and is designed so that its base(its inner arc), is at least equal to this minimum dimension. The heightof the wedge in the radial direction is always equal to the minimumdimension. Each 4-bit data symbol is represented by an array of 2×2wedges. The fifteen 4-bit data symbols of each of the six codewords areallocated to the four concentric symbol rings 18 a to 18 d ininterleaved fashion. Symbols are allocated alternately in circularprogression around the tag. The interleaving is designed to maximise theaverage spatial distance between any two symbols of the same codeword.

In order to support “single-click” interaction with a tagged region viaa sensing device, the sensing device must be able to see at least oneentire tag in its field of view no matter where in the region or at whatorientation it is positioned. The required diameter of the field of viewof the sensing device is therefore a function of the size and spacing ofthe tags. Assuming a circular tag shape, the minimum diameter of thesensor field of view 193 is obtained when the tags are tiled on aequilateral triangular grid, as shown in FIG. 4 b.

The tag structure just described is designed to allow both regulartilings of planar surfaces and irregular tilings of non-planar surfaces.Regular tilings are not, in general, possible on non-planar surfaces. Inthe more usual case of planar surfaces where regular tilings of tags arepossible, i.e. surfaces such as sheets of paper and the like, moreefficient tag structures can be used which exploit the regular nature ofthe tiling.

An alternative tag structure more suited to a regular tilting is shownin FIG. 5 a. The tag 4 is square and has four perspective targets 17. Itis similar in structure to tags described by Van de Grift et al. in U.S.Pat. No. 5,051,746. The tag represents sixty 4-bit Reed-Solomon symbols47, for a total of 240 bits. The tag represents each one bit as a dot48, and each zero bit by the absence of the corresponding dot. Theperspective targets are designed to be shared between adjacent tags, asshown in FIGS. 5 b and 5 c. FIG. 5 b shows a square tilting of 16 tagsand the corresponding minimum field of view 193, which must span thediagonals of two tags. FIG. 5 c shows a square tilting of nine tags,containing all one bits for illustration purposes.

Using a (15, 7) Reed-Solomon code, 112 bits of tag data are redundantlyencoded to produce 240 encoded bits. The four codewords are interleavedspatially within the tag to maximize resilience to burst errors.Assuming a 16-bit tag ID as before, this allows a region ID of up to 92bits. The data-bearing dots 48 of the tag are designed to not overlaptheir neighbors, so that groups of tags cannot produce structures whichresemble targets. This also saves ink. The perspective targets thereforeallow detection of the tag, so further targets are not required.

Although the tag may contain an orientation feature to allowdisambiguation of the four possible orientations of the tag relative tothe sensor, it is also possible to embed orientation data in the tagdata. For example, the four codewords can be arranged so that each tagorientation contains one codeword placed at that orientation, as shownin FIG. 5 d, where each symbol is labelled with the number of itscodeword (1–4) and the position of the symbol within the codeword (A-O).Tag decoding then consists of decoding one codeword at each orientation.Each codeword can either contain a single bit indicating whether it isthe first codeword, or two bits indicating which codeword it is. Thelatter approach has the advantage that if, say, the data content of onlyone codeword is required, then at most two codewords need to be decodedto obtain the desired data. This may be the case if the region ID is notexpected to change within a stroke and is thus only decoded at the startof a stroke. Within a stroke only the codeword containing the tag ID isthen desired. Furthermore, since the rotation of the sensing devicechanges slowly and predictably within a stroke, only one codewordtypically needs to be decoded per frame.

It is possible to dispense with perspective targets altogether andinstead rely on the data representation being self-registering. In thiscase each bit value (or multi-bit value) is typically represented by anexplicit glyph, i.e. no bit value is represented by the absence of aglyph. This ensures that the data grid is well-populated, and thusallows the grid to be reliably identified and its perspective distortiondetected and subsequently corrected during data sampling. To allow tagboundaries to be detected, each tag data must contain a marker pattern,and these must be redundantly encoded to allow reliable detection. Theoverhead of such marker patterns is similar to the overhead of explicitperspective targets. One such scheme uses dots positioned a variouspoints relative to grid vertices to represent different glyphs and hencedifferent multi-bit values (see Anoto Technology Description, AnotoApril 2000).

Decoding a tag results in a region ID, a tag ID, and a tag-relative pentransform. Before the tag ID and the tag-relative pen location can betranslated into an absolute location within the tagged region, thelocation of the tag within the region must be known. This is given by atag map, a function which maps each tag ID in a tagged region to acorresponding location. A tag map reflects the scheme used to tile thesurface region with tags, and this can vary according to surface type.When multiple tagged regions share the same tiling scheme and the sametag numbering scheme, they can also share the same tag map. The tag mapfor a region must be retrievable via the region ID. Thus, given a regionID, a tag ID and a pen transform, the tag map can be retrieved, the tagID can be translated into an absolute tag location within the region,and the tag-relative pen location can be added to the tag location toyield an absolute pen location within the region.

The tag ID may have a structure which assists translation through thetag map.

It may, for example, encoded cartesian coordinates or polar coordinates,depending on the surface type on which it appears. The tag ID structureis dictated by and known to the tag map, and tag IDs associated withdifferent tag maps may therefore have different structures.

Two distinct surface coding schemes are of interest, both of which usethe tag structure described earlier in this section. The preferredcoding scheme uses “location-indicating” tags as already discussed. Analternative coding scheme uses “object-indicating” (or“function-indicating”) tags.

A location-indicating tag contains a tag ID which, when translatedthrough the tag map associated with the tagged region, yields a uniquetag location within the region. The tag-relative location of the pen isadded to this tag location to yield the location of the pen within theregion. This in turn is used to determine the location of the penrelative to a user interface element in the page description associatedwith the region. Not only is the user interface element itselfidentified, but a location relative to the user interface element isidentified. Location-indicating tags therefore trivially support thecapture of an absolute pen path in the zone of a particular userinterface element.

An object-indicating (or function-indicating) tag contains a tag IDwhich directly identifies a user interface element in the pagedescription associated with the region (or equivalently, a function).All the tags in the zone of the user interface element identify the userinterface element, making them all identical and thereforeindistinguishable. Object-indicating tags do not, therefore, support thecapture of an absolute pen path. They do, however, support the captureof a relative pen path. So long as the position sampling frequencyexceeds twice the encountered tag frequency, the displacement from onesampled pen position to the next within a stroke can be unambiguouslydetermined. As an alternative, the netpage pen 101 can contain a pair ormotion-sensing accelerometers, as described in our earlier applicationU.S. Ser. No. 09/575,174.

With either tagging scheme, the tags function in cooperation withassociated visual elements on the netpage as user interactive elementsin that a user can interact with the printed page using an appropriatesensing device in order for tag data to be read by the sensing deviceand for an appropriate response to be generated in the netpage system.

Each application user interface flow is illustrated as a collection ofdocuments linked by command arrows. A command arrow indicates that thetarget document is printed as a result of the user pressing thecorresponding command button on the source page. Some command arrows arelabelled with multiple commands separated by slashes (‘/’s), indicatingthat any one of the specified commands causes the target document to beprinted. Although multiple commands may label the same command arrow,they typically have different side-effects.

In application terms, it is important to distinguish between netpagedocuments and netpage forms. Documents contain printed information, aswell as command buttons which can be pressed by the user to requestfurther information or some other action. Forms, in addition to behavinglike normal documents, also contain input fields which can be filled inby the user. They provide the system with a data input mechanism. It isalso useful to distinguish between documents which contain genericinformation and documents which contain information specific to aparticular interaction between the user and an application. Genericdocuments may be pre-printed publications such as magazines sold at newsstands or advertising posters encountered in public places. Forms mayalso be pre-printed, including, for example, subscription formsencountered in pre-printed publications. They may, of course, also begenerated on-the-fly by a netpage printer in response to user requests.User-specific documents and forms are normally generated on the fly by anetpage printer in response to user requests. FIG. 6 shows a genericdocument 990, a generic form 991, a user-specific document 992, and auser-specific form 993.

Netpages which participate in a user interface flow are furtherdescribed by abstract page layouts. A page layout may contain variouskinds of elements, each of which has a unique style to differentiate itfrom the others. As shown in FIG. 7, these include fixed information994, variable information 995, input fields 996, command buttons 997,draggable commands 998, and text hyperlinks or hypertext links 999.

When a user interface flow is broken up into multiple diagrams, anydocument which is duplicated is shown with dashed outlines in all butthe main diagram which defines it.

8. Device Controls using the Netpage™ System

The netpage system, then provides the facility to control electrical orelectronic devices. If a particular device can be remotely controlled,either via a network interface or an infrared interface, then it ispossible to completely separate the user interface to the device fromthe device itself. Furthermore, the user interface can then beimplemented as a netpage, sensed by a netpage sensing device carried bythe user. The netpage sensing device should ideally be small,unobtrusive and portable. The netpage pen provides one possible formfactor. Alternatively, the netpage sensing device might be embedded in aring, hand glove or finger glove worn by the user. It could also beembedded in custom eyewear, with interaction controlled by gaze.

The netpage identifies the device, and is printed with the userinterface of the device in the form of buttons etc. Being passive, thesurface giveds no feedback to the user as to the status of the device orits user interface. Feedback can, however, be provided via the sensingdevice if necessary. The netpage base station (e.g. the netpage Printer)resolves operations on the netpage with reerence to a user interfacestructure previosuly selected by the user from the device itself or itsmanufacturer, or designed by the user using elements provided by thedevice or its manufacturer.

The following illustrates the concepts of a netpage device control indetail by way of a netpage device control application for audio andvideo equipment. Although specific reference is made to a netpage pen,any netpage sensing device can be used. Similarly, although specificreference is made to a netpage printer, any netpage base station can beused, i.e. any device which receives input from a netpage sensing deviceand either interprets the input directly or forwards the input to one ormore network servers for interpretation.

Device Control

The netpage system provides a mechanism for controlling devices such astelevision receivers, VCRs and video-on-demand units.

A netpage can contain printed control buttons in the style of a remotecontrol unit, or printed playback control buttons associated withindividual programs or movies. Playback controls can include play,pause, stop, start and end. An iconic image of the program or movie canitself act as a play button.

Consumer disk-based video recorders are now available which supportsimultaneous recording and playback. They allow buffered viewing ofbroadcast video and provide some of the advantages of video-on-demand. Adaily newspaper delivered via the netpage system can contain links tobroadcast news video captured in this way some time before delivery ofthe newspaper, perhaps under control of the netpage printer.

With broadcast television, a printed netpage program listing can providea useful interface to a collection of television channels, particularlyif one or more channels are being buffered by a disk-based videorecorder. With true video-on-demand, a printed netpage catalog canprovide a useful interface to a video library.

The netpage pen can be augmented to work as an infrared remote control.Commands generated by pen interactions with printed video playbackcontrols can also be sent directly from the netpage printer to the videodevice, by wired or wireless control interface, including infrared andradio (e.g. Bluetooth).

In the netpage system device control services are provided by one ormore device control applications running on servers distributed on thenetwork. User interactions with device control netpages are routed tothe appropriate device control application via the appropriate pageserver using normal netpage mechanisms.

Various device control interaction scenarios are illustrated in FIG. 8.When a user invokes a device function on a device control netpage 500,the input is sent from the user's netpage pen 101 to the user's netpageprinter 802, in the usual way (A). The input is relayed to theapplication 503 in the usual way (B). On receipt of the input, theapplication identifies the designated device 600 and determines thespecified device command. If the designated device 600 is directlyaccessible to an application, then the application may send the commandcode directly to the device (C1). This may be the case if the device isconnected to a network accessible to the application 503, e.g. a wiredor wireless wide-area network. Otherwise the application may send thecommand code to the originating printer 802 (C2). The printer 802 may bemodified to be receptive to device control commands sent to it by adevice control application. If the designated device 600 is directlyaccessible to the printer 802, then the printer may send the commandcode directly to the device (D1). This may be the case if the device isconnected to a network accessible to the printer, e.g. a wired orwireless wide-area or local-area network, or if the printer and devicecan communicate directly by wire or wirelessly. Otherwise the printer502 may send the command code to the originating pen 101 (D2). The pen101 may be modified to be receptive to device control commands sent toit by a printer 502. The pen may be augmented with an infraredtransmitter, allowing it to transmit the command code to the device(E1). This requires the pen's infrared transmitter to be in line ofsight of the device's infrared receiver. As an alternative to sendingthe command code to the pen 101, the printer 502 may instead send thecommand code to a remote control “pod” 504 (D3). This is a device whichis installed within line of sight of the device 600. It can communicatewith the printer by wire or wirelessly, and incorporates an infraredtransmitter, allowing it to transmit the command code to the device 600(E2). Because the pod 504 may be placed in a location which gives it aguaranteed view of a number of devices, it provides a useful alternativeto augmenting the pen 101.

Device Control Object Model

The Device Control object model revolves around devices and devicesetups maintained for particular netpage users in relation to particularnetpage printers. The Device Control class diagram is shown in FIG. 9.

Each netpage user 800 has one or more device setups 511. A user may havea separate device setup 511 for each netpage printer 802 they haveaccess to. For example, a user 800 may have one device setup 511 at homeand another at their office. Each device setup 511 has a name 512assigned by the user. Each device setup 511 may have one or more users,but each user 800 only has access to one device setup 511 per printer802.

A device setup 511 has a number of user devices 513 associated with it,one for each device 600 the user can control. Each user device 513 hasan alias name 514 assigned by the user, and has associated with it themanufacturer's device model information.

Each device 513 can have a number of remote controls 515. A remotecontrol can be used by any user 800 authorized for the device setup 511.A remote control 515 has a number of remote control functions 510associated with it, each with a function identifier 517. The functionidentifier 517 is used to look up the control code 565 for the functionfrom the manufacturer 568. A remote control 515 may be device specific518, universal 519 (i.e. control more than one device), or generic 520.The Remote Control class diagram is shown in FIG. 13.

A device-specific remote control 518 only operates the user device forwhich it was printed.

A universal remote control (URC) 519 may operate several devices withinthe device setup 511. The user selects a device 513 on the URC 519 tochange the URC mode and thereby to choose which device is current. TheURC 519 typically only operates one device in each mode.

A generic remote control 520 can operate any device 569 for any user. Ageneric remote control 520 may be printed in a netpage publication suchas a television program. A generic remote control 520 operates thedefault devices 521 in the device setup 511. If, for example, ahousehold has more than one television the remote control 520 will onlyoperate the default television. For each device type, if no defaultdevice 521 has been specified by the user then the first device of therelevant type added by the user is automatically made the default. Adefault video receiver 522 may be specified for a specific broadcastchannel 523. This can be useful where one channel can be received onseveral of the user's devices 513. The Default Device class diagram isshown in FIG. 10.

When adding a device 569 to their device setup 511 the user 800 mustspecify the manufacturer 568, device type and model of the device 524.

Each manufacturer 568 has a number of devices 569 and each device 569has a number of models 524. Each device model 524 has associated with ita remote control interface 525, and information about the device'schannel, date, time and length of program (CDTL) programming capability526. The remote control interface 525 contains information about theremote control functions 516 available for the device 569. Each remotecontrol interface (RCI) function 516 has a function identifier 527 andthe control code 565 to perform the function. The CDTL capability 526specifies the information that can be programmed for the device. Forexample, a VCR can be programmed with CDTL information, whereas a cabletelevision receiver may only be able to be programmed with channel, dateand time (CDT) information. The Device information class diagram isshown in FIG. 11.

Each device setup 511 may have associated with it a number of programrequests 568. A program request 568 is created when a user 800 requestsa video broadcast program to be recorded. A published television programschedule 528 contains a number of entries 529, one for each televisionprogram to be broadcast. Associated with each entry is one or more CDTLinformation 530 records containing the channel, date, time and programlength information for the broadcast. This information can then be usedto control the user devices 513 for unattended recording of broadcasttelevision programs. The Program Schedule class diagram is shown in FIG.15.

The Device Channel class diagram is illustrated in FIG. 13. Each device569 may have associated with it channel locality information 531,describing the broadcast channels available within a given locality.These may be divided into channel categories 532, e.g. free-to-air,cable, etc.

The user may receive a given broadcast channel 533 through several oftheir devices, for example free-to-air broadcasts may be availablethrough the television, VCR, and cable box. Optionally, the user has theability to define a default video receiver device for each channel 533.If the user elects, via a program schedule, to record a program on achannel which is available through several of the user's devices, thenthe default video receiver, as specified in the user's device setup 511,is used.

Device Control User Interface

The user 800 typically obtains the netpage Device Setup main page 534from the Device Control application's entry in the netpage systemapplication catalog, obtained via the netpage printer's help page. Theuser may also bookmark the page and subsequently obtain it via theirfavorites list, or obtain it via their history list.

The Device Control user interface flow is shown in FIG. 16.

Device Setup Main Page

The Device Setup main page 534, shown in FIG. 21, lists the devicesetups for the current netpage printer 802, and the users registered foreach setup. If the privileges setting on the printer is not enabled thenall device setups for the printer are listed. If the privileges settingis enabled then only those device setups that the user is authorized tosee are listed. If the user is a system administrator then all devicesetups are listed.

To print the details for a particular device setup the user presses the<Details> button 535 next to the desired setup. To create a new devicesetup, the user enters a setup name 536 and presses the <New DeviceSetup> button 537. Both actions print a Device Setup Details page 538.

The user can press <List My Devices> 539 to print a Device List page 540for the user's own device setup. The Device Setup user interface flow isshown in FIG. 16.

The Device Setup Details page 538, shown in FIG. 22, lists all theregistered users and devices for a device setup. If the device setup isnew, then no registered users or devices are listed.

The user can print the full details of a registered user or device bypressing the <Details> 541 button next to the desired entry. For a userentry a User Details page 542 is printed. For a device entry a DeviceDetails page 543 is printed showing the device manufacturer, devicetype, model, device alias, and the default devices to which this deviceis assigned.

The user can remove a user or device from the device setup 511 bypressing the <Remove> button 544 next to the appropriate entry. TheDevice Setup Details page 538 is reprinted with the appropriate entryremoved.

The user can add a new user to the device setup by pressing the <Add NewUser> button 545. An Add New User form 546 is printed listing all usersregistered on the current printer but not included in any device setup.A user can only be included in one device setup per netpage printer.

The user can add a new device to the device setup by pressing the <AddNew Device> button 547. A Select Manufacturer page 548 is printed.

The Setup Details user interface flow is shown in FIG. 17.

Adding a New Device

A new device 513 can be added to a device setup 511 by pressing the <AddNew Device> button 547 on the Device Setup Details page 538. A SelectManufacturer page 548 is printed, as shown in FIG. 27. The user selectsthe device manufacturer 549 from the list by clicking the name ofmanufacturer. A Select Device Type page 550 is printed, as shown in FIG.28.

The user selects the appropriate device type 551 from the list byclicking on the device type name. A Select Model page 552 is printed, asshown in FIG. 29.

The user selects the appropriate device model 553 from the list byclicking on the model name. The Set Device Defaults form 554 is printed,as shown in FIG. 30.

On the Set Device Defaults form 554 the user specifies an alias name forthe device. The user then assigns this device as the default for one ormore device types by checking the appropriate check boxes. For example,the user's VCR may be assigned as the default television tuner anddefault video recorder in their device setup. When the form is completethe user presses <Add Device> 555. The Device Setup Details page 538 isprinted, showing the new device in the device list. The Add Device userinterface flow is shown in FIG. 19.

User's Device List

The user typically obtains the User's Device List 540 from the DeviceSetup main page 534 (<List My Devices> button 539). The user may alsobookmark the page and subsequently obtain it via their favorites list,or obtain it via their history list.

The Device List user interface flow is shown in FIG. 18.

The Device List 540 is shown in FIG. 23 and indicates which devices areassigned as defaults for the setup. To print a Remote Control page 570for a specific device the user clicks on the device name. An example ofa device-specific television remote control 570 is shown in FIG. 25. Adevice-specific remote control 570 will only operate the device named atthe top of the remote control. A device-specific remote control for aparticular television will only operate that one television, and willnot operate any other television.

The user can press the <Create URC> button 556 to print a CreateUniversal Remote Control form 557, as shown in FIG. 24. This form listsall the devices in the current device setup. The user selects thedevices to include on the URC by checking the appropriate device checkboxes, and then clicking the <Print URC> button 558. The UniversalRemote Control page 571 is printed. An example of a URC page 571 isshown in FIG. 26.

The user can press the <Device Setup> button 559 on the Device List Page540 to print their Device Setup Details page 538.

The user can press the <Modify Defaults> button 560 to print a ModifyDefaults form 561. This form allows the user to change the assignment ofdefault devices.

The user can press the <List Requests> button 562 to print a RequestList page 563. This lists all the video broadcasts scheduled to berecorded for the user's device setup.

Remote Controls

There are three types of remote control. These are the device-specificremote control, the universal remote control, and the generic remotecontrol. Device specific and universal remote controls are printed bythe user. Generic remote controls are printed in netpage publicationssuch as television program schedules.

The appearance and functions of a remote control will depend on thedevice type, manufacturer etc. A manufacturer may provide their ownnetpage remote control which can be printed by the user on demand.

When the user presses a remote control button on a generic remotecontrol the device control application looks up the appropriate defaultdevice in the user's device setup. This determines the user device andtherefore the device model.

A universal remote control 571 is illustrated in FIG. 26. The devicebeing controlled by a universal remote control is determined by its mode(set when a device alias name button 564 is pressed on the control). Adevice-specific remote control 570 as shown in FIG. 25 is associatedwith a single user device. Thus when the user presses a remote controlbutton 565 on a device-specific or universal remote control the devicealias name, and therefore the device model, of the device beingcontrolled is known.

A remote control button 565 is associated with a remote control function510, with a unique function identifier 517 (unique within the devicemodel). The device model is referenced in the manufacturer's database tolook up the RCI (Remote Control Interface) function 516 control code 565for the required function identifier. This command code is thentransmitted to the user device.

The method of the transmission may vary depending on the device'scapability and the user's pen.

Scheduled Broadcast Recording

A netpage published television program schedule 572 contains a calendarof television broadcasts giving a description 566 of each program, andchannel, date, and time information about the program. Each programlisting may have associated with it one or more buttons 567, as shown inFIG. 31.

The <Record> button 567 can be used to program the user's default videorecording device to record the requested broadcast. The request 568 isstored in the user's program request list 573 (associated with theirdevice setup). The Record Program user interface flow is shown in FIG.20.

As a more advanced service, a program information provider can sendupdated program information to the user's printer, or can add programrequests to the user's program request list 573 on a regular basis for auser's favorite programs. It can also automatically modify existingrequests if program information changes.

News Clips

A published news story can have an associated video clip and printedplayback control buttons.

A set of news clips can be broadcast some time before the newspaper isdelivered to the netpage printer 802. The printer 802 can be instructedby the publisher to listen to the broadcast and record the broadcast onthe user's disk-based video recording system. Clip indexing informationis transmitted with the video clips, for example in the closed captionpart of the broadcast.

The buttons printed with the news story control playback of thebroadcast video clip from the user's default disk-based video recorderto the user's default video monitor. Control buttons may include play,stop, slow motion, and freeze frame.

Another type of video clip is the video-on-demand (VOD) video clip. AVOD video clip is broadcast to the user on demand, from a third partyVOD supplier.

Parental Control

The Netpage device control system can also be used to give parentalcontrol over rated content. The system administrator can set theallowable content rating for particular users. A user without therequired authority cannot operate a remote control to view ratedchannels or broadcasts.

CONCLUSION

The present invention has been described with reference to a preferredembodiment and number of specific alternative embodiments. However, itwill be appreciated by those skilled in the relevant fields that anumber of other embodiments, differing from those specificallydescribed, will also fall within the spirit and scope of the presentinvention. Accordingly, it will be understood that the invention is notintended to be limited to the specific embodiments described in thepresent specification, including documents incorporated bycross-reference as appropriate. The scope of the invention is onlylimited by the attached claims.

1. A method of enabling a device to be controlled via a printed controlinterface, the control interface comprising user control instructionsrelating to the device and the control interface including invisiblecoded data and visible graphic data printed substantially simultaneouslythereon by a single printer wherein the invisible coded data isindicative of an identity of the control interface and of coordinates ofa plurality of reference points of the control interface, the coded dataidentifying a unique location of each of the reference points relativeto the control interface, and wherein when the single printer prints theinvisible coded data and the visible graphic data on the controlinterface substanitally simulataneously, a computer system automaticallyassociates a type and spatial extent of each reference point of theinvisible coded data with a spatial extent of at least some of thevisible graphic data in which the device is operative to perform atleast one function in response to control instructions from the computersystem, the method including the steps of: receiving, in the computersystem, indicating data from an optical sensing device regarding theidentity of the control interface and a position of the sensing devicerelative to the control interface, the sensing device, when placed in anoperative position relative to the control interface, reading at leastsome of the coded data on the control interface, and generating theindicating data using at least some of the read coded data; andeffecting, in the computer system and from the indicating data, anoperation relating to at least one parameter of the controlinstructions.
 2. The method of claim 1 in which said at least oneparameter relating to the control instructions is associated with atleast one zone of the control interface and in which the method includeseffecting, in the computer system and from the zone relative to whichthe sensing device is located, an operation relating to said at leastone parameter.
 3. The method of claim 2 which includes receiving, in thecomputer system, data regarding movement of the sensing device relativeto the control interface, the sensing device sensing its movementrelative to the control interface using at least some of the coded data;and effecting, in the computer system and from said movement being atleast partially within said at least one zone, an operation relating tosaid at least one parameter of the control instructions.
 4. The methodof claim 1 in which the parameter of the control instructions isselected from the group comprising: selecting said device; selectingsaid function to be performed; establishing default setting for saidfunction; establishing default setting for said device; registering useraccess to control the device function; authorizing user access tocontrol the device function; and issuing a command code to said deviceto perform said function.
 5. The method according to claim 1 furtherincluding the step of issuing a command code to said device to performsaid function in response to operation of the computer system.
 6. Themethod of claim 5 in which the command code is issued to said devicethrough said sensing device.
 7. The method of claim 5 in which thecommand code is issued to said device independently of said sensingdevice.
 8. The method according to claim 5 in which the command code isissued to said device using wireless technology.
 9. The method of claim1 which includes printing the control interface on demand.
 10. Themethod of claim 9 which includes printing the control interface on asurface of a surface-defining means and, at the same time that thecontrol interface is printed, printing the coded data on the surface.11. The method of claim 1 which includes retaining a retrievable recordof each control interface generated, the control interface beingretrievable using its identity as contained in its coded data.
 12. Themethod of claim 1 which includes distributing a plurality of the controlinterfaces using a mixture of multicast and pointcast communicationsprotocols.
 13. The method of claim 1 in which the sensing devicecontains an identification means which imparts a unique identity to thesensing device and identifies it as belonging to a particular user andin which the method includes monitoring, in the computer system, saididentity.
 14. The method of claim 1 which includes providing allrequired information relating to the device function in the controlinterface to eliminate the need for a separate display device.
 15. Themethod of any one of claim 1 in which the control interface is printedon multiple pages and in which the method includes binding the pages.16. A system for enabling at least one function of a device to becontrolled, the system including: a printed control interface comprisinguser control instructions relating to the device and coded dataindicative of an identity of the control interface and of coordinates ofa plurality of locations on the control interface and includinginvisible coded data and visible graphic data printed substantiallysimultaneously thereon by a single printer, wherein the invisible codeddata is indicative of an identity of the control interface and of aplurality of coordinates of reference points of the control interface,the coded data identifying a unique location of each of the referencepoints relative to the control interface, and wherein when the singleprinter prints the invisible coded data and the visible graphic data onthe control interface substanitally simulataneously, a computer systemautomatically associates a type and spatial extent of each referencepoint of the invisible coded data with a spatial extent of at least someof the visible graphic data; and a computer system incorporating controlinstructions operative to cause said device to perform said function andoperative to receive indicating data from an optical sensing device foreffecting an operation relating to at least one parameter of the controlinstructions, the indicating data being indicative of the identity ofthe control interface and a position of the sensing device relative tothe control interface, the sensing device reading the coded data on thecontrol interface and generating the indicating data using at least someof the read coded data.
 17. The system of claim 16 in which said atleast one parameter relating to the control instructions is associatedwith at least one zone of the control interface.
 18. The system of claim16 which includes the sensing device sensing its movement relative tothe control interface using at least some of the coded data.
 19. Thesystem of claim 16 in which the parameter of the control instructions isselected from the group comprising: selecting said device; selectingsaid function to be performed; establishing default setting for saidfunction; establishing default setting for said device; registering useraccess to control the device function; authorizing user access tocontrol the device function; and issuing a command code to said deviceto perform said function.
 20. A system according to claim 16 in whichthe computer is operative to issue a command code to said device toperform said function through said sensing device.
 21. A systemaccording to claim 16 in which the computer is operative to issue acommand code to said device to perform said function independently ofsaid sensing device.
 22. The system of claim 16 in which the sensingdevice contains an identification means which imparts a unique identityto the sensing device and identifies it as belonging to a particularuser.
 23. The system of claim 16 in which the control interface isprinted on a surface of a surface-defining means and in which the systemincludes a printer for printing the control interface on demand.
 24. Thesystem of claim 23 in which the printer prints the coded data at thesame time as printing the control interface on the surface-definingmeans.
 25. The system of claim 16 which includes a database for keepinga retrievable record of each control interface generated, each controlinterface being retrievable by using its identity as included in itscoded data.
 26. The system of claim 23 in which, to cater for a controlinterface printed on multiple pages, the printer includes a bindingmeans for binding the pages.